Repository navigation
Trim the encoder delay and padding of AAC ADTS streams - #157
Open
chrizzchrizz wants to merge 5 commits into
Open
chrizzchrizz wants to merge 5 commits into
chrizzchrizz wants to merge 5 commits into
Conversation
Decoders output the encoder delay (priming) and padding (remainder) of formats like AAC and MP3 as audio. The packet table info was only used for the duration, so consecutive entries were separated by a short silence (2112 + remainder frames, about 50 ms, for AAC). - Keep only the decoded frames in [priming, priming + valid) while the decoded position is exact (decoding from the start of the stream) - After a seek, drop the remainder at the end of the stream instead - Frame counts are converted to the output sample rate
- Mark the decoded position as inexact when re-reading from a seek offset (reset(decodedPositionExact:)), so mid-stream data isn't trimmed as the start - Fall back to trailing-only trimming when packet table info arrives late - Account for frames after a fully filled decode so they aren't overwritten - Pass the EOF entry to trimTrailingGaplessPadding instead of looking it up - Extract the trim math into GaplessTrim and add unit tests
- Keep the decoded position exact when seeking to the start of the data - Ignore repeated packet table info once the gapless values are set - Round the end of the kept range once to avoid an off-by-one with resampling
An MP3 without Xing/Info header reports no packet table info: a transcode of a server (live or from its cache), or a file of a simple encoder. The decoder then outputs the encoder delay at the start and the padding at the end as audio, so two entries are separated by a gap of about 35 ms, which is a tick in a continuous mix. The trimming added for AAC and tagged MP3 does not apply. - Assume the encoder delay of LAME (576 frames, the decoder delay is already compensated by AudioToolbox) for an MP3 that reports no packet table info when it is ready to produce packets. The packet table info of a file with header is reported before and takes precedence. - The padding is unknown. At the end of the stream drop the frames at the end of the renderer buffer that are below 20 percent of the peak level of the music before them (at most one frame plus the encoder delay, and only frames that are not played yet). The codec rings for a few dozen frames after music that ends abruptly, so a fixed silence threshold is not enough. Measured with the end of one track and the start of the next one of a DJ mix (M4A transcoded to MP3 by Navidrome, with and without its transcoding cache), played back to back and compared sample by sample with the lossless source: gap at the transition 1535 frames (cached) / 659 frames (live) -> 0 frames, on macOS and on an iPhone 15 Pro. A sine cut in two parts, MP3 files with and without Xing header, and RAW: no gap, no phase jump.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Trim the encoder delay and padding of AAC ADTS streams
Problem
AAC-LC in an ADTS stream cannot carry gapless info: there is no packet table and no iTunSMPB. This is the format servers usually stream when they transcode to AAC live. Navidrome, for example, uses ffmpeg
-c:a aac -f adtsand serves it asaudio/aacwith no Content-Length. The decoder then plays the encoder delay at the start and the padding at the end as audio. Between two entries of a continuous mix this gives a gap of about 1640 samples (1024 priming plus the end padding), which you hear as a tick.Changes
This applies the approach of #150 for MP3 without Xing/Info header to ADTS:
kAudioFileAAC_ADTSType) reports no packet table info, assume the encoder delay of ffmpeg's native AAC encoder (1024 frames). AAC in MP4 and HE-AAC are not affected.assumedPrimingFramesnow also takes the file type, which is read fromkAudioFileStreamProperty_FileFormat.Testing
GaplessTrimTests: priming is assumed for AAC in ADTS, but not for AAC in MP4, HE-AAC in ADTS or other formats. The maximum padding is checked per format.swift test: all tests pass..aacfile. That matches the MP3 result of Trim the encoder delay and padding of MP3 streams without gapless info #150.